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(57) Abstract 

Disclosed is a device and method for use in conjunction 
with a computer (34) for providing commands to a computer 
(34). A stylus (11) is coupled to a mechanical linkage (14 
and 17) which is in turn supportable on a fixed surface (23). 
The mechanical linkage (14 and 17) or the stylus (11) is in 
communication with sensors (13A, 13B, 16A, 16B, 19A, 19B) 
for sensing the orientation of the stylus (11). An ancillary 
remote command control unit (25 or 26) is also disclosed 
herein. 
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THREE-DIMENSIONAL MECHANICAL MOUSE 

FIELD OF THE INVENTION 
The present invention relates to a computer-human interface device, and more 
particularly it relates to a stylus coupled to a supportable mechanical linkage for 
providing commands to a computer. 

BACKGROUND OF THE INVENTION 

As the use of Computer Aided Design (CAD) Systems becomes more 
widespread, the need for cursor and command control devices which accurately and 
easily track three-dimensional position or motion is also growing. Devices which allow 
users to control a cursor with three-dimensional position and/or orientation commands 
are available for various applications. Among them are many hand-held input devices 
which allow users to interact with a host processor by controlling the position of a 
cursor or manipulating graphic objects on a computer screen. While these devices 
allow three-dimensional information to be transmitted to a computer they do not allow 
the user to use gestures and motions which are natural to the user. 

For example, a prior art device of the type which is used for three-dimensional 
control involves the use of accelerometers to transduce the position and orientation of 
a stylus in space as described in U.S. PaL No. 4.839,838. This device makes no 
provisions so the stylus can be grasped in a manner which makes use of finger 
dexterity nor does it include mechanical support to reduce fatigue or enhance user 
control or dexterity. 
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Another prior art example is an ultrasonic position-locating device like the one 
shown in U.S. Pat No. 5,142,506. This device transduces position and orientation by 
triangulating ultrasonic signals. As with the prior art previously described, this device 
uses a free-floating stylus which includes no provisions for mechanical support to 
reduce fatigue or enhance user control or dexterity. Furthermore, this device is used 
with a stylus that is grasped in the palm of the hand. The use of such a stylus 
precludes fine positioning with the fingers and greatly reduces the dexterity of the user 
to manipulate position and orientation. In addition, this device is used with digital 
buttons on the stylus to send to the computer command signals. A button of this type 
is commonly called a "clicker 41 on a "mouse." Because such buttons are mechanically 
coupled to the free-floating stylus, it is difficult to push the buttons while maintaining 
the position and orientation of the stylus. By pushing down on the button, the user 
will necessarily move the stylus from its desired position. Accordingly, these 
commands are difficult to control under many circumstances. 

SUMMARY OF THE INVENTION 

In the present invention, the user holds a stylus which is supported by a 
support apparatus on a fixed surface so that the user can easily manipulate the 
stylus in free space to interact with a computer. The three-dimensional motion of the 
user is translated through the stylus and mechanical linkage to a processor which 
communicates with the computer, thus allowing commands to be sent to the computer 
which track the three-dimensional motion of the user. Therefore, cursor control in 
three-dimensions on the two-dimensional computer screen is possible. 

In one embodiment, the stylus is supportable on a fixed surface by a set of 
mechanical linkages which include individual components joined together by a 
sufficient number of joints to allow several degrees of freedom in the motion of the 
stylus. These mechanical linkages provide mechanical leverage, friction, 
counter-weighing, and/or spring resistance in order to reduce fatigue of the user and 
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to provide support to enhance the stability and dexterity of user manipulation of the 
stylus. 

In the aforementioned embodiment of the present invention, the joints of the 
mechanical linkages are coupled to sensors which provide information about their 
position. Such information is transmitted to a microprocessor so that position and 
orientation of the stylus can be computed using kinematic equations associated with or 
related to the particular linkage system. In another embodiment, position and 
orientation of the stylus is sensed through the use of ultrasonic, magnetic, or optical 
position and orientation sensors mounted on the stylus. 

Another aspect of the present invention includes a remote control unit which is 
used in place of a command clicker on the stylus. For example, a foot pedal or hand- 
held unit for the user's opposite hand is included to provide command control to the 
computer. Accordingly, manual dexterity of stylus manipulation is not compromised. 



SUBSTITUTE SHEET (RULE 26) 



WO 95/02801 FCT/US94/07851 

4 

DESCRIPTION OF THE DRAWINGS 
Figure 1 is a perspective view of an embodiment of the present invention; 

Figures 2A and 2B are block diagrams over-viewing two different electronic 
hardware configurations of the present invention; 

Figure 3 is a flow chart describing the main software loops for two different 
electronic hardware configurations shown in FIG. 2; 

Figures 4A and 4B are flow charts describing two different interrupt service 
routines for serial output to host computer; 

Figure 5 is a perspective representation of another embodiment of the present 
invention; 

Figure 6 is a perspective view of still another embodiment of the present 
invention; 

Figure 7 is a perspective representation of another embodiment; 

Figure 8 is a perspective view of another embodiment 

Figure 9 shows an embodiment of the resistance mechanism of the present 
invention; 

Figure 10 shows another embodiment of the resistance mechanism; and 
Figure 11 shows yet another embodiment of the resistance mechanism. 
DESCRIPTION OF PREFERRED EMBODIMENTS 
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Referring to FIG. 1, a stylus 11 is shown attached to a support apparatus 
which is, in turn, supported on a fixed surface. By electrical and electronic 
configurations described below, the stylus 11 is adapted to provide data from which a 
computer or other computing means such as a microprocessor can ascertain the 
5 position and orientation of the stylus as it moves in three-dimensional space. This 

information is then translated to an image on a computer display apparatus. The 
stylus 11 may be used, for example, by an operator to change the position of a cursor 
on a computer controlled display screen by changing the position and/or orientation of 
the stylus, the computer being programmed to change the position of the cursor in 
10 proportion to the change in position and/or orientation of the stylus. In other words, 

the stylus 11 is moved through space by the user to designate to the computer how 
or where to move the cursor on a computer display apparatus. 

The stylus 11 is a pen-like stick which can be manipulated between the 
fingers, allowing for much better control and fine dexterity as compared to full hand 
grips or palm-supported styluses used by some prior art inventions. While the stylus 
11 is described in terms of manual manipulation, other stylus configurations are 
envisioned by the present invention. In particular, this invention includes manipulation 
by those unable to manually manipulate a pen. A stylus of the present invention, need 
not be linear, but may be curved or angled so that it may be held, for example, by the 
foot or the mouth of a person. 

Because the stylus is supported by a support apparatus which Is in turn 
supported by a fixed surface or other stabilizing configuration, the user can manipulate 
the stylus with a minimum of effort. Also, if the user chooses to discontinue using the 
stylus, it is capable of maintaining its position in space, unattended. While FIG. 1 
. 25 shows that preferred embodiment of the present invention, FIGS. 5-8 show alternative 

embodiments, such which are also contemplated under the present invention. It is 
preferable that the stylus have enough degrees of freedom to enable it to move 
through the mechanical linkage to give the user the amount of flexibility needed to 
move the cursor as desired. In FIG. 1, six degrees of freedom are shown and are 
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labeled as Axis' 16. This, of course, provides maximum flexibility. Fewer degrees of 
freedom, such as a plurality of degrees of freedom, may also be sufficient depending 
on the application. 

In one embodiment, the stylus is connected to rigid individual components 
5 which are joined together by joints. While not shown, other types of support 

apparatus' are included in the present invention. For example, other configurations 
include a semi-flexible rod or any other moveable while supportive configuration which 
can support the stylus in the manner described herein. 

In FIG. 1, a mechanical linkage pursuant to the present invention is depicted. 
The stylus 11 is coupled to supportable mechanical linkages via joint 12 which, in the 
shown embodiment, houses sensors 13A and 13B. Linkage 14, is connected, via 
joint 15 having position sensors 16A and 16B, to linkage 17. Joint 18 in turn 
connects linkage 17 with the vertical base protrusion 20 which emanates from the 
base 21. The sensors are used to produce a stylus locative signal which is 
responsive to and corresponds with the position of the stylus at any point in time 
during its normal operation. The stylus locative signal is used to provide information 
for use by a computer display apparatus of a computer. The term "joint" as used 
herein is intended to mean the connection mechanism between individual linkage 
components. In fact, two separate moveable members can be joined; such together 
forming a joint 

The base 21, if necessarily, can be immobilized by securing it onto the fixed 
surface 23 by way of bolt, screw or other attachment mechanism 22. Moreover, the 
present invention implements mechanical leverage and rubbing friction (not shown) 
between the supportable mechanical linkages 14 and 17 and the joints 12, 15 and 18 
25 in order to provide resistance and support so as to allow better dexterity than can be 

achieved with free-floating stylus trackers. This support and leverage aids in reducing 
the fatigue associated with manipulating the free-floating stylus 11. 
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As mentioned above, attached to each joint 12, 15 and 18 are sensors 13A, 
13B, 16A, 16B, 19A, and 19B, respectively. These sensors sense the angle differential 
before and after motion of the two segments connected by that joint The sensors can 
be, for example, optical incremental encoders, optical absolute encoders and 
potentiometers. Because the three-dimensional position and/or orientation tracking is 
achieved mechanically, this preferred embodiment avoids problems that magnetic and 
ultrasonic sensors, such as those shown in the prior art, encounter with metal and 
shadowing. However, as shown in FIG. 1, if desired, sensing means can be used to 
track the position and/or orientation of the stylus by mounting a single or several 
orientation sensors in the stylus 11 itself, such referred to as a stylus mounted 
sensor 11. An ultrasound, magnetic, optical or position and orientation sensor can be 
used as the stylus mounted sensor 11. 

FIG. 1 also shows a clicker button 24 on stylus 11. The button is connected 
to a switch which when in the on state, sends a signal to the computer giving it a 
command. In order to provide for accuracy when sending commands, this invention 
also includes a remote clicker unit. Therefore, since the clicking motion occurs at a 
distant location from the cursor control, there is little or no opportunity to accidently 
move the cursor while making a command. FIG. 1 shows two configurations for 
implementing this aspect of the present invention. The first is identified as an alternate 
hand-clicker 25, the second as foot pedal 26. 

Digital buttons 27 and 28 which are connected to switches (not shown) on the 
remote attached peripherals such as a hand-held clicker unit 25 or a foot pedal 26, 
respectively, can generate additional digital input such transmitted through lines 25' 
and 26' respectively. Either of the shown ancillary remote command units, such 
including the hand unit 25 and the foot pedal 26 configurations, are favorable methods 
of inputting digital commands by command hardware or software (not shown) because 
pressing the button 27 or 28 does not compromise a user's ability to hold the stylus 
steady whereas pressing any button 24 on the stylus does compromise stylus stability. 
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Referring to FIG. 2A. the sensors 13A, 13B, 16A, 16B. 19A and 19B, along 
with any peripherals 24, 25 or 26, can send their digital signals directly to a versatile 
floating-point processor or microprocessor 32A which is controlled by software stored 
in a digital ROM (Read-Only Memory) via transmission line 32' or another form of 
transmission, i.e., radio signals. As shown in FIG. 2B, an alternative embodiment can 
be used to lessen the demands on the floating-point processor or microprocessor 32B. 
The digital inputs of the sensors 13A, 13B, 16A, 16B, 19A and 19B can be sent 
indirectly to the floating-point processor or microprocessor 32B by way of dedicated 
chips 13C, 13D, 16C, 16D, 19C and 19D, which pre-process the angle sensors' 
signals before sending them via bus 31 to the floating-point processor or 
microprocessor 32B which would combine these signals with those from the 
peripherals 24, 25 or 26. An 8-bit data bus plus chip-enable lines allow any of the 
angle determining chips to communicate with the microprocessor. Moreover, reporting 
the status of peripherals 24, 25 or 26 includes reading the appropriate digital switch 
and placing its status in the output sequence array. Some examples of specific 
electronic hardware usable for sensor pre-processing include quadrature counters, 
which are common dedicated chips that continually read the output of an optical 
incremental encoder and determine an angle from it, Gray decoders, filters, and ROM 
look-up tables. 

The single-chip configuration of FIG. 2A is most applicable where the angle 
sensors 13A, 13B, 16A, 16B, 19A and 19B are absolute sensors, which have output 
signals directly indicating the angles without any further processing, thereby requiring 
less computation for the microprocessor 32A and thus little if any pre-processing. The 
multi-chip configuration of FIG. 2B is most applicable if the sensors 13A, 13B, 16A, 
16B, 19A and 19B are relative sensors, which indicate oniy the change in an angle and 
which require further processing for complete determination of the angle. 

In either configuration, it the microprocessor 32A or 32B is fast enough, it will 
compute stylus 11 position and/or orientation (or motion, if desired) on board the 
embodiment and send this final data through any standard communications interface 
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such as an RS-232 serial interface 33 on to the host computer system 34 and to 
computer display apparatus 34" through transmission line 34' or another form of 
transmission. If the microprocessor 32A or 32B is not fast enough, then the angles 
will be sent to the host computer 34 which will perform these calculations on its own. 

In addition to the single-chip and multi-chip configurations, a variation may 
consist of a single microprocessor which reads the peripherals, obtains the angles, 
possibly computes coordinates and orientation of the stylus 11, and supervises 
communication with the host computer 34. Another variation may consist of dedicated 
sub-circuits and specialized or off-the-shelf chips which reads the peripherals, 
monitors the angle sensors 13A, 13B, 16A, 16B, 19A and 19B, determine the joint 
angles, and handle communications with the host computer 34, all without software or 
a microprocessor 32A or 32B. 

Software is only included in the two microprocessor-based configurations 
shown in FIGS. 2A and 2B. The more dedicated hardware a given configuration 
includes, the less software it requires. The software consists of a main loop (FIG. 3) 
and an output interrupt (FIGS. 4A and 4B). 

Referring to FIG. 3, the main command loop responds to the host computer 34 
and runs repeatedly in an endless cycle. With each cycle, incoming commands from 
the host computer are monitored 36 and decoded 37, and the corresponding 
command subroutines for reporting angles, thus stylus position and/or orientation (see 
FIGS. 4A and 4B), are then executed 38. Two possible subroutines are shown in 
FIGS. 4A and 4B. When a subroutine terminates, the main command loop resumes 
39. Available command will include but are not limited to: reporting the value of any 
single angle, reporting the angles of all six angles at one time, reporting the values of 
all six angles repeatedly until a command is given to cease aforementioned repeated 
reporting, reporting the status of peripheral buttons, and setting communications 
parameters. If the angle sensors require preprocessing, these commands will also 
include resetting the angle value of any single angle or otherwise modifying 
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preprocessing parameters in other applicable ways. Resetting pre-processed angle 
values or preprocessing parameters does not require output data from the device. The 
microprocessor 32A or 326 simply sends appropriate control signals to the 
preprocessing hardware 13C, 130, 16C, 16D, 19C, and 190. If the microprocessor or 
5 floating-point processor is fast enough to computer stylus coordinates and orientation, 

these commands will also include reporting the stylus coordinates once reporting the 
stylus coordinates repeatedly until a command is given to cease, ceasing 
aforementioned repeated reporting, reporting the stylus coordinates and orientation 
once, reporting the stylus coordinates and orientation repeatedly until a command is 
10 given to cease, ceasing aforementioned repeated reporting. If force reflection is 

supported, these commands will also include reporting the forces felt by any single 
joint, setting the resistance of any single joint and locking or unlocking a joint 

Any report by the subroutines of FIGS. 4A and 4B of a single angle value 
requires determining 41 the given joint angle. For the single-chip configuration shown 
in FIG. 2A, this subroutine directly reads the appropriate angle sensor 42 from among 
sensors 13A, 13B, 16A, 16B, 19A, and 19B. For the multi-chip configuration shown in 
FIG. 2B, this subroutine reads the outputs 43 of pre-processing hardware 13C, 13D, 
16C, 16D, 19C, and 19D which have already determined the joint angles from the 
outputs of the sensors 13A, 13B, 16A, 16B, 19A, and 19B. Any report of multiple 
angles is accomplished by repeatedly executing the subroutine for reporting a single 
angle. The subroutine is executed once per angle, and the values of all angles are 
then included in the output sequence array. If the optional parts of the subroutines 45 
are included, then these subroutines become the coordinate reporting subroutines. 
Many other commend subroutines exist and are simpler yet in their high-level structure. 

25 After determining the given joint angle, the microprocessor 32A or 32B creates 

an output sequence 44A or 44B by assembling an array in a designated area of 
processor memory which will be output by the microprocessor's communications 
system at a given regular communications rate. The sequence will contain enough 
information for the host computer 34 to deduce which command is being responded 
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to, as well as the actual angle value that was requested. Returning to FIG. 3, a query 
36 in the main command loop asks whether the previous command requested repeated 
reports. If so, the main command loop is initiated accordingly. The communications 
output process (not shown) may be as simple as storing the output data in a 
designated output buffer, or it may involve a standard set of communications interrupts 
that are an additional part of the software. Setting communications parameters does 
not require output data from the device. The microprocessor 32A or 32B simply 
resets some of its own internal registers or sends control signals to its 
communications sub-unit. 

To report the stylus' 1 1 coordinates, three of the five or six angle values 
pre-read and knowledge of link lengths and device kinematics are incorporated to 
compute stylus 11 coordinates. These coordinates are then assembled in the output 
sequence array. 

To report the stylus' 1 1 orientation, at least five angle values are read and 
knowledge of link lengths and device kinematics are incorporated to computer 
stylus 11 orientation. The orientation consists of three angles (not necessarily 
identical to any joint angles) which are included in the output sequence array. 

Forces felt by a joint, setting a joint's resistance, and locking or unlocking a 
joint are reported by using interaction of the microprocessor 32A or 32B with 
forced-reflecting hardware (not shown). Reporting forces felt by a joint uses a force 
sensor mounted on the joint and then places the resulting value in the output sequence 
array. To set a joint's resistance and locking or unlocking a joint control signals 
reading from a force sensor to force-reflection hardware but do not require any output 
data of the device. 

Also contemplated in the present invention is computer software and hardware 
. which will provide feedback information from the computer to the stylus (not shown). 
This type of implementation is known in robotics and thus is easily incorporated into a 
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system including the present indention. When a surface is generated on the computer 
screen, the computer will send feedback signals to the mechanical linkage which has 
force generators for generating force in response to the cursor position on the surface 
depicted on the computer screen. Force is applied for example, by added tension in 
the joints which is in proportion to the force being applied by the user and in 
conjunction with the image on the screen. 

The various configurations of the mechanical linkages shown in FIG. 5, FIG. 6, 
FIG. 7 and FIG. 8 which have different numbers of individual components and joints 
than shown in FIG. 1 are Illustrative of the numerous possible configurations which can 
provide varying degrees of freedom inherent in the present invention. Referring to FIG. 
5, FIG. 6 and FIG. 8, note that a rounded object such as a ball can act as a joint 
having motion in three degrees of freedom. In conjunction with other mechanical 
linkages and attachments, this permits sufficient degrees of freedom for the purposes 
of the present invention. In each figure, the orientation of the degrees of freedom of 
each joint is depicted by curved lines, numbered consecutively. 

Briefly, FIG. 5 shows a rounded joint 46 at the base such that three degrees of 
motion are available at that joint FIG. 6 shows a three-dimensionally rotatable 
rounded joint 47 at the base through which one mechanical linkage can slide linearly 
and where the base is attached to a fixed surface 48 such that the surface does not 
prohibitively impede the movement of the device. FIG. 7 shows an embodiment where 
the basal connection can slide about the base in a two-dimensional plane in the cross 
configuration 49 on base 51. FIG. 8 shows a three-dimensionally rotatable rounded 
joint 52 at a perpendicular projection from the base 53 through which one mechanical 
linkage 54 can slide linearly through the joint 52. 

While any of the above discussed configurations or others can be used in 
accordance with the present invention, FIGS. 9-11 show different mechanisms for 
providing resistance to the manual manipulation of the stylus by the user. FIG. 9, for 
example, shows return or tension springs 56 on each joint of the embodiment shown 
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in FIG. 1. In an alternative embodiment FIG. 10, shows counter-weights 57 on each 
joint. Moreover, FIG. 11. shows a combination of a return or tension spring 56, a 
counter-wight 57 and a compression spring 58. The arrangement of the resistance 
mechanism used should depend upon the configuration stylus mechanical linkage 
5 combination, such arrangement preferably chosen to maximize the ease with which the 

user can manipulate the stylus 11 in free space in accordance with the present 
invention. 
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WHAT IS CLAIMED : 

1 . A device tor use in conjunction with a computer display apparatus and 
a fixed surface, comprising: 

a stylus; 

means supportable on a fixed surface and coupled to said stylus for 
supporting said stylus while allowing at least a plurality of degrees of freedom in the 
motion of said stylus; and 

means for producing a stylus locative signal responsive to and 
corresponding with the position of the stylus at any point in time during its normal 
operation, said stylus locative signal providing information for use by a computer 
display apparatus. 

2. A device as recited in Claim 1 wherein supporting means is a 
mechanical linkage. 

3. A device as recited in Claim 2 wherein said stylus locative signal 
means is in communication with said mechanical linkage. 

4. A device as recited in Claim 1 wherein said stylus locative signal 
means is in communication with said stylus. 

5. A device as recited in Claim 1 further comprising: 

a remote unit having switch capable of being in an on state and an off 

state; and 

command means triggered by said switch when said switch is in its on 
state for generating a command signal for receipt by a computer. 

6. A device as recited in Claim 5 wherein said remote unit is a foot pedal 

unit. 
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7. A device as recited in Claim 2 wherein said mechanical linkage includes 
three individual components. 

8. A device as recited in Claim 2 wherein said mechanical linkage includes 
at least three joints. 

9. A device as recited in Claim 1 further comprising means for providing 
resistance to the motion of the stylus. ' 

10. A device as recited in Claim 1 wherein said stylus has pencil-like 
configuration which can be manually manipulated. 

11. A device as recited in Claim 1 further comprising: 
feedback means for generating force by said support means in 

response to force signals provided to said device, said force signals correlated to 
information displayed on computer display apparatus. 

12. A method for interfacing a user and a computer display apparatus, 
comprising the steps of: 

providing a stylus; 

coupling to said stylus a support apparatus supportable on a fixed 
surface for supporting said stylus while allowing at least a plurality of degrees of 
freedom in the motion of said stylus; and 

providing means for producing a stylus locative signal responsive to 
and corresponding with the position of the stylus at any point in time during its normal 
operation, said stylus locative signal providing information for use by a computer 
display apparatus. 

13. A method as recited in Claim 12 wherein support apparatus is a 
mechanical linkage. 
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14. A method as recited in Claim 13 wherein said stylus locative signal 
means is in communication with said mechanical linkage. 

15. A method as recited in Claim 12 wherein said stylus locative signal 
means is in communication with said stylus. 

16. A method as recited in Claim 12 further comprising the steps of: 
providing a remote unit having switch capable of being in an on state 

and an off state; and 

providing a command signal generator triggered by said switch when 
5 said switch is in its on state for generating a command signal for receipt by a 

computer. 

17. A method as recited in Claim 16 wherein said remote unit is a foot 
pedal unit. 

18. A method as recited in Claim 13 wherein said mechanical linkage 
includes three individual components. 

19. A method as recited in Claim 13 wherein said mechanical linkage 
includes at least three joints. 

20. A method as recited in Claim 12 further comprising means for 
providing resistance to the motion of the stylus. 

21. A method as recited in Claim 12 wherein said stylus has pencil-like 
configuration which can be manually manipulated. 

22. A device as recited in Claim 11 further comprising the steps of: 
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providing feedback means for generating force by said support means 
in response to force signals provided to said device, said force signals correlated to 
information displayed on computer display apparatus. 

23. In a computer system having a main unit and cursor control functions 
and command control functions which are triggered remotely from said main unit, a 
device for interfacing a user and computer, comprising: 

a remote command control unit separate from a cursor control unit 
having a switch capable of being in an on state and an off state; 

command means triggered by said switch when said switch is in its on 
state for generating a command signal representative of a command from a user to a 
computer; and 

transmission means for transmitting said command signal to said 

computer. 

24. A device as recited in Claim 21 wherein said remote command control 
unit is a foot pedal unit 

25. In a computer system having a main unit and cursor control functions 
and command control functions which are triggered remotely from said main unit, a 
system for interfacing a user and computer comprising the steps of: 

providing a remote command control unit separate from a cursor 
control unit having a switch capable of being in an on state and an off state; 

generating a command signal representative of a command from a user 
to a computer when said switch is in its on state; and 

transmitting said command signal to said computer. 

26. A system as recited in Claim 23 wherein said remote command control 
unit is a foot pedal unit. 
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